Papers with internal prediction construction process
LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Transformer-based language models (LMs) are opaque and unexplained, causing problems for endusers and developers who wish to debug or fix their behaviour. |
| Approach: | They propose an interactive debugger tool for transformer-based LMs that provides a fine-grained interpretation of the model's internal prediction process and a powerful framework for intervening in LM behavior. |
| Outcome: | The proposed tool provides a fine-grained interpretation of the model's internal prediction construction process, and a powerful framework for intervening in LM behavior. |
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Fig. 1 shows how feed-forward network (FFN) layers are utilized to build LMs. |
| Approach: | They reverse-engineer the operation of feed-forward network layers to find out how they work . they show that each update can be decomposed to sub-updates corresponding to single parameter vectors . |
| Outcome: | The proposed model reduces the toxicity of GPT2 by almost 50% and improves computation efficiency with a simple early exit rule, saving 20% of computation on average. |
Understanding Transformer Memorization Recall Through Idioms (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing memorization use definitions that are based on model performance, which changes between models and often also between training runs. |
| Approach: | They propose idioms as inputs that typically trigger memory recall and propose a set of English idiomas to test their methodological framework for probing and characterizing recall of memorized sequences in transformer LMs. |
| Outcome: | The proposed framework compares model behavior on memorized vs. non-memorized inputs across different model sizes and architectures. |